1. Diagnostics
1.1 General Diagnostics
When a vehicle component begins to fail, the repair cost is frequently minimal if the impending failure of the component is caught early, but increases as the repair is delayed. Sometimes, if a component in need of repair is not caught in a timely manner, the component, and particularly the impending failure thereof, can cause other components of the vehicle to deteriorate. One example is where the water pump fails gradually until the vehicle overheats and blows a head gasket. Another example is when a tire gradually loses air until it heats up, fails and causes an accident. It is desirable, therefore, to determine that a vehicle component is about to fail as early as possible so as to minimize the probability of a breakdown and the resulting repair costs.
There are various gages on an automobile which alert the driver to various vehicle problems. For example, if the oil pressure drops below some predetermined level, the driver is warned to stop his vehicle immediately. Similarly, if the coolant temperature exceeds some predetermined value, the driver is also warned to take immediate corrective action. In these cases, the warning often comes too late as most vehicle gages alert the driver after he or she can conveniently solve the problem. Thus, what is needed is a component failure warning system that alerts the driver to the impending failure of a component sufficiently in advance of the time when the problem gets to a catastrophic point.
Some astute drivers can sense changes in the performance of their vehicle and correctly diagnose that a problem with a component is about to occur. Other drivers can sense that their vehicle is performing differently but they don't know why or when a component will fail or how serious that failure will be, or possibly even what specific component is the cause of the difference in performance. The invention disclosed herein will, in most cases, solve this problem by predicting component failures in time to permit maintenance and thus prevent vehicle breakdowns.
Presently, automobile sensors in use are based on specific predetermined or set levels, such as the coolant temperature or oil pressure, whereby an increase above the set level or a decrease below the set level will activate the sensor, rather than being based on changes in this level over time. The rate at which coolant heats up, for example, can be an important clue that some component in the cooling system is about to fail. There are no systems currently on automobiles to monitor the numerous vehicle components over time and to compare component performance with normal performance. Nowhere in the vehicle is the vibration signal of a normally operating front wheel stored, for example, or for that matter, any normal signal from any other vehicle component. Additionally, there is no system currently existing on a vehicle to look for erratic behavior of a vehicle component and to warn the driver or the dealer that a component is misbehaving and is therefore likely to fail in the very near future.
Basically, the operating of an automobile should be a process not a project. A purpose of this invention is to eliminate breakdowns through identifying potential component failures before they occur so that they can be repaired in a timely manner. Another purpose is to notify the operator and a service facility of the pending failure so that it can be prevented.
Sometimes, when a component fails, a catastrophic accident results. In the Firestone tire case, for example, over 100 people were killed when a tire of a Ford Explorer blew out which caused the Ford Explorer to rollover. Similarly, other component failures can lead to loss of control of the vehicle and a subsequent accident. It is thus important to accurately forecast that such an event will take place but furthermore, for those cases where the event takes place suddenly without warning, it is also important to diagnose the state of the entire vehicle, which in some cases can lead to automatic corrective action to prevent unstable vehicle motion or rollovers resulting in an accident.
Finally, an accurate diagnostic system for the entire vehicle can determine much more accurately the severity of an automobile crash once it has begun by knowing where the accident is taking place on the vehicle (e.g., the part of or location on the vehicle which is being impacted by an object) and what is colliding with the vehicle based on a knowledge of the force deflection characteristics of the vehicle at that location. Therefore, in addition to a component diagnostic, the teachings of this invention also provide a diagnostic system for the entire vehicle prior to and during accidents. In particular, this invention is concerned with the simultaneous monitoring of multiple sensors on the vehicle so that the best possible determination of the state of the vehicle can be determined. Current crash sensors operate independently or at most one sensor may influence the threshold at which another sensor triggers a deployable restraint as taught in the current assignee's U.S. patent application Ser. No. 10/638,743 filed Aug. 11, 2003 and related patents and pending applications. In the teachings of this invention, two or more sensors, frequently accelerometers, are monitored simultaneously and the outputs of these multiple sensors can be combined continuously in making the crash severity analysis.
U.S. Pat. No. 5,754,965 (Hagenbuch) describes an apparatus for diagnosing the state of health of a construction vehicle and providing the operator of the vehicle with a substantially real-time indication of the efficiency of the vehicle in performing as assigned task with respect to a predetermined goal. A processor in the vehicle monitors sensors that provide information regarding the state of health of the vehicle and the amount of work the vehicle has done. The processor records information that describes events leading up to the occurrence of an anomaly for later analysis. The sensors are also used to prompt the operator to operate the vehicle at optimum efficiency. The system of this patent does not predict or warn the operator or the home base of a pending problem.
Asami et al. (U.S. Pat. No. 4,817,418) is directed to a failure diagnosis system for a vehicle including a failure display means for displaying failure information to a driver. This system only reports failures after they have occurred and does not predict them.
Tiernan et al. (U.S. Pat. No. 5,313,407) is directed, inter alia, to a system for providing an exhaust active noise control system, i.e., an electronic muffler system, including an input microphone 60 which senses exhaust noise at a first location 61 in an exhaust duct 58. An engine has exhaust manifolds 56,57 feeding exhaust air to the exhaust duct 58. The exhaust noise sensed by the microphone 60 is processed to obtain an output from an output speaker 65 arranged downstream of the input microphone 61 in the exhaust path in order to cancel the noise in the exhaust duct 58. No attempt is made to diagnose system faults nor predict them.
Haramaty et al. (U.S. Pat. No. 5,406,502) describes a system that monitors a machine in a factory and notifies maintenance personnel remote from the machine (not the machine operator) that maintenance should be scheduled at a time when the machine is not in use. Haramaty et al. does not expressly relate to vehicular applications.
NASA Technical Support Package MFS-26529 “Engine Monitoring Based on Normalized Vibration Spectra”, describes a technique for diagnosing engine health using a neural network based system but does not suggest that this system can or should be used on land vehicles.
A paper “Using acoustic emission signals for monitoring of production processes” by H. K. Tonshoff et al. also provides a good description of how acoustic signals can be used to predict the state of machine tools and is incorporated by reference herein in its entirety. Again no suggestion is made that this can be used for diagnosing components of land vehicles.
1.2 Pattern Recognition
Marko et al. (U.S. Pat. No. 5,041,976) is directed to a diagnostic system using pattern recognition for electronic automotive control systems and particularly for diagnosing faults in the engine of a motor vehicle after they have occurred. For example, Marko et al. is interested in determining cylinder specific faults after the cylinder is operating abnormally. More specifically, Marko et al. is directed to detecting a fault in a vehicular electromechanical system directly, i.e., by means of the measurement of parameters of sensors which are designed to be affected only by that system, and after that fault has already manifested itself in the system. In order to form the fault detecting system, the parameters from these sensors are input to a pattern recognition system for training thereof. Then, known faults are introduced and the parameters from the sensors are input into the pattern recognition system with an indicia of the known fault. Thus, during subsequent operation, the pattern recognition system can determine the fault of the electromechanical system based on the parameters of the sensors, assuming that the fault was “trained” into the pattern recognition system and has already occurred.
When the electromechanical system is an engine, the parameters input into the pattern recognition system for training thereof, and used for fault detection during operation, all relate to the engine. In other words, each parameter will be affected by the operation of the engine and depend thereon and changes in the operation of the engine will alter the parameter, e.g., the manifold absolute pressure is an indication of the airflow into the engine. In this case, the signal from the manifold absolute pressure sensor may be indicative of a fault in the intake of air into the engine, e.g., the engine is drawing in too much or too little air, and is thus affected by the operation of the engine. Similarly, the mass air flow is the airflow into the engine and is an alternative to the manifold absolute pressure. It is thus a parameter that is directly associated with, related to and dependent on the engine. The exhaust gas oxygen sensor is also affected by the operation of the engine, and thus directly associated therewith, since during normal operation, the mixture of the exhaust gas is neither rich or lean whereas during abnormal engine operation, the sensor will detect an abrupt change indicative of the mixture being too rich or too lean.
Thus, the system of Marko et al. is based on the measurement of sensors which affect or are affected by, i.e., are directly associated with, the operation of the electromechanical system for which faults are to be detected. However, the system of Marko et al. does not detect faults in the sensors that are conducting the measurements, e.g., a fault in the exhaust gas oxygen sensor, or faults that are only developing but have not yet manifested themselves or faults in other systems. Rather, the sensors are used to detect a fault in the system after it has occurred. Marko does not attempt to forecast or predict that a fault will occur.
Aside from the references above of assignee's patents and patent applications and the one example of an engine control system, pattern recognition has not been applied to the diagnosis of any faults on a vehicle. In the referenced examples, the engine controller for example, only sensors directly associated with the component have been used. No attempt has been made to forecast that a failure will occur and no system has been disclosed other than by the assignee for transmitting such diagnostic information to a site off of the vehicle.
1.3 SAW and Other Wireless Sensors in General
Surface Acoustic Wave (SAW) and other wireless sensors have particular advantages in sensing vehicle and component parameters as will now be discussed.
One of the first significant SAW sensor patents is U.S. Pat. No. 4,534,223. This patent describes the use of SAW devices for measuring pressure and also a variety of methods for temperature compensation but does not mention wireless transmission.
One method of measuring pressure that is applicable to this invention is disclosed in V. V. Varadan, Y. R. Roh and V. K. Varadan “Local/Global SAW Sensors for Turbulence”, IEEE 1989 Ultrasonics Symposium p. 591-594 makes use of a polyvinylidene fluoride (PVDF) piezoelectric film to measure pressure. This article discloses that other piezoelectric materials can also be used. Experimental results are given where the height of a column of oil is measured based on the pressure measured by the piezoelectric film used as a SAW device. In particular, the speed of the surface acoustic wave is determined by the pressure exerted by the oil on the SAW device. For the purposes herein, air pressure can also be measured in a similar manner by first placing a thin layer of a rubber material onto the surface of the SAW device which serves as a coupling agent from the air pressure to the SAW surface. In this manner, the absolute pressure of a tire, for example, can be measured without the need for a diaphragm and reference pressure greatly simplifying the pressure measurement. Other examples of the use of PVDF film as a pressure transducer can be found in U.S. Pat. Nos. 4,577,510 and 5,341,687, which are incorporated by reference herein, although they are not used as SAW devices.
In recent years, SAW devices have been used as sensors in a broad variety of applications. Compared with sensors utilizing alternative technologies, SAW sensors possess important properties such as high sensitivity, high resolution, and ease of manufacturing by microelectronic technologies. However, the most attractive feature of SAW sensors is that they can be interrogated wirelessly and that they can be operated without a battery or other source of power except for the RF signal that is captured by the antenna. SAW devices, however, have a very low signal strength which will now be discussed.
1.3.1 Signal Boosting
In the use of SAW sensors for vehicles, one problem arises from vehicle vibrations that can interfere with or create excessive noise in the signals provided by the SAW sensor due to the generally low strength of the signal from the SAW sensor. In many cases for SAW tire monitors, for example, an adequate return signal can be obtained while the vehicle is stationary but the signal degrades as the vehicle moves. Thus, whereas the device can operate without power in the stationary mode it is desirable to have a source of power when the vehicle is moving. However, when the vehicle is moving there is a significant amount of energy available in the vehicle tire, and elsewhere in the environment, to permit the powered operation of the SAW device. This is known herein as signal boosting. Such signal boosting, as described below, can increase the gain by as much as 6 db in both directions, or a total of 12 db, or more. The Energy generated can be stored on a capacitor, or ultracapacitor, or on a rechargeable battery as appropriate. There appears to be no known prior art on this concept.
Heretofore, RF MEMS switches have not been used in the tire, RFID or SAW sensor environment such as for TPM power and antenna switching as disclosed herein. Such RF-MEMS switches can be advantageously used with a booster circuit. PCT application WO03047035A1 “GPS equipped cellular phone using a SPDT MEMS switch and single shared antenna” does describe such a use for cell phones. One example of an RF MEMS switch is manufactured by Teravicta Technologies Inc. The company's initial product, the TT612, is a 0 to 6 GHz RF MEMS single-pole, double-throw (SPDT) switch. It has a loss of 0.14-dB at 2-GHz, good linearity and a power handling capability of three watts continuous, all enclosed within a surface mount package.
Teravicta claims the RF performance of its switch is superior to that of conventional solid-state alternatives such as gallium arsenide FETs and PIN diodes that are used in today's wireless voice and data products.
1.3.2 Energy Generation
Several prior art patents have disclosed the use of various non-battery power sources for the use with tire monitors. These include inductive, capacitive and generator systems using a moving weight. Other systems that are disclosed herein and assignee's copending patent applications to charge an energy storage device use an RFID circuit, the earth's magnetic field with a coil, a solar sensor, a MEMS or other energy generator that uses the vibrations in the tire and a generator that uses the bending deflection of tread or the deflection of the tire itself relative to the tire rim as sources of energy. These can be used with the boosting circuit with or without a MEMS RF or other appropriate mechanical or electronic switch.
1.3.3 Communication, ID
The combination of an RFID with a SAW device has also not been reported in the prior art. This combination in addition to providing energy to boost the SAW system can also provide a tire identification to the interrogator. The ID portion of the RFID can be in the form of a SAW Polyvinylidene Fluoride RFID Tag that can be manufactured at low cost or using a conventional memory. The use of such a PVDF SAW RFID has not heretofore been reported.
1.3.4 Miscellaneous Enhancements
A SAW Pressure Sensor can also be used for many pressure sensing applications such as bladder weight sensors permitting that device to be interrogated wirelessly and without the need to supply power. This also can use the boosting techniques as disclosed herein. Similarly, a SAW device can be used as a general switch in a vehicle and in particular as a seatbelt Buckle switch indicative of seatbelt use.
The operating frequency of SAW devices has heretofore been limited to less than about 500 MHz due to manufacturing problems. However, recent advances in the manufacture of diamond films that can be combined with a piezoelectric material such as lithium niobate promise to permit higher frequencies to be used.
The use of a SAW based TPM (tire pressure monitor) as described herein, and particularly and boosted SAW based TPM as described herein, permits the after market replacement for other battery powered TPM systems such as those manufactured by Schrader with the replacement product removing the need for a battery and thus periodic replacement.
The use of a piezoelectric generator attached to the tire tread also gives means to measure the tire footprint and thus a measure of the load on the car and truck tires. This can also be accomplished by the system that is powered by the change in distance between the tread and the rim as the tire rotates coupled with a measure of the pressure within the tire. There appears to be no prior art for either concept.
In a different but related invention disclosed below for the first time, the driver is provided with a keyless entry device that can be powerless in the form of an RFID or similar device, that can also be boosted as described herein, and the vehicle mounted interrogator determines the proximity of the driver to the vehicle door. If the driver remains within 1 meter from the door, for example, for a time period of 5 seconds, for example, then the door automatically unlocks and ever opens in some implementations. Thus as the driver approaches the trunk with his or her arms filled with groceries and pauses, the trunk can automatically open. Such a system would be especially valuable for older people. Naturally this system can also be used for other systems in addition to vehicle doors and trunk lids.
Another novel implementation is to place a SAW or RFID transponder in the vehicle key and prevent the doors from locking if the keys are inside unless the engine is running or a driver is present. This would eliminate the accidental locking of the keys inside the vehicle that has been one of the main uses of the OnStar system by subscribers.
1.4 Tire Monitoring
In August, 2000, Bridgestone/Firestone Inc. recalled approximately 6.5 million Firestone ATX, ATX II and Wilderness AT tires used primarily on Ford Motor Co. light trucks and sport utility vehicles, including Ford's best-selling Explorer. The National Highway Traffic Safety Administration (NHTSA) is investigating Firestone tires in connection with approximately 150 U.S. traffic deaths and more than about 400 injuries. Most of the Firestone tire deaths occurred when the tires came apart while on Ford Explorers, causing the vehicles to roll over.
Bridgestone/Firestone has been criticized for not ordering a recall sooner, even though the company's data on claims for injuries and property damage indicated problems with the tires at least as early as 1997. Ford received harsh criticism after the Firestone recall because it acknowledged ordering its own recall of the same tires in 16 other countries after receiving reports of problems. The foreign recalls began more than a year before the U.S. recall, but Ford never alerted NHTSA. Ford was not required by law to report the foreign recalls.
Spurred in particular by the recent problems with Firestone tires, the U.S. House of Representatives passed a bill requiring vehicle rollover testing and installation of systems to warn of under-inflated tires. It allows for stiff prison sentences for automotive industry executives who hide safety problems. According to the bill, there could be a 15-year sentence for officials who withhold information on defective products from government investigators. It also includes a safe harbor provision that would allow whistle-blowers to report defects within a reasonable amount of time without being punished. Moreover, companies would have to tell NHTSA about tire recalls overseas.
The House bill also requires that all vehicles have warning indicators for low tire pressure and it includes a provision requiring NHTSA to develop driving tests to determine vehicle rollover risk instead of the simple mathematical formula the agency plans to use.
It is known to use internal-to-vehicle mechanisms for monitoring the air pressure of the tires of a vehicle. These mechanisms have a stationary device which interacts with a device that co-moves with the respective wheel of the vehicle in such a way that monitoring of the air pressure can take place during operation of the vehicle. The co-moving device uses suitable means to sense the air pressure, and transmits an output-related signal to the stationary device if the air pressure falls below a certain value. A prerequisite for operation of these systems is that the co-moving device has an energy supply, for example a small battery rotating along with the wheel being monitored. This configuration must therefore be included in ongoing maintenance cycles so that a battery exchange is performed at the proper time. The battery exchange leads to additional costs. Moreover, the mass of the rotating wheel is influenced by the requisite battery device; in particular, an asymmetrical mass distribution results, which requires additional counterweights. Overall wheel balance is therefore degraded.
Tire monitoring is now extremely important since NHTSA (National Highway Traffic Safety Administration) has recently linked 148 deaths and more than 525 injuries in the United States to separations, blowouts and other tread problems in Firestone's ATX, ATX II and Wilderness AT tires, 5 million of which were recalled in 2000. Many of the tires were standard equipment on the Ford Explorer. Ford recommends that the Firestone tires on the Explorer sport utility vehicle be inflated to 26 psi, while Firestone recommends 30 psi. It is surprising that a tire can go from a safe condition to an unsafe condition based on an under inflation of 4 psi as suggested by Firestone.
According to a NHTSA research survey, 27 percent of passenger cars on U.S. roadways are driven with one or more substantially under-inflated tires. In addition, the survey found that 33 percent of light trucks (including sport utility vehicles, vans and pickup trucks) are driven with one or more substantially under-inflated tires.
Recent studies in the United States conducted by the Society of Automotive Engineers show that low tire pressure causes about 260,000 accidents annually. Another finding is that about 75% of tire failures each year are preceded by slow air leaks or inadequate tire inflation. Nissan, for example, warns that incorrect tire pressures can compromise the stability and overall handling of a vehicle and can contribute to an accident. Additionally, most non-crash auto fatalities occur while drivers are changing flat tires. Thus, tire failures are clearly a serious automobile safety problem that requires a solution.
About 16% of all car accidents are a result of incorrect tire pressure. Thus, effective pressure and wear monitoring is extremely important. Motor Trend magazine stated that one of the most overlooked maintenance areas on a car is tire pressure. An estimated 40 to 80 percent of all vehicles on the road are operating with under-inflated tires. When under-inflated, a tire tends to flex its sidewall more, increasing its rolling resistance which decreases fuel economy. The extra flex also creates excessive heat in the tire that can shorten its service life.
The Society of Automotive Engineers reports that about 87 percent of all flat tires have a history of under-inflation. About 85% of pressure loss incidents are slow punctures caused either by small-diameter objects trapped in the tire or by larger diameter nails. The leak will be minor as long as the nail is trapped. If the nail comes out, pressure can decrease rapidly. Incidents of sudden pressure loss are potentially the most dangerous for drivers and account for about 15% of all cases.
A properly inflated tire loses approximately 1 psi per month. A defective time can lose pressure at a more rapid rate. About 35 percent of the recalled Bridgestone tires had improper repairs.
Research from a variety of sources suggests that under-inflation can be significant to both fuel economy and tire life. Industry experts have determined that tires under-inflated by a mere 10% wear out about 15% faster. An average driver with an average set of tires can drive an extra 5,000 to 7,000 miles before buying new tires by keeping the tire properly inflated.
The American Automobile Association has determined that under inflated tires cut a vehicle's fuel economy by as much as 2% per psi below the recommended level. If each of a car's tires is supposed to have a pressure of 30 psi and instead has a pressure of 25 psi, the car's fuel efficiency drops by about 10%. Depending on the vehicle and miles driven that could cost from $100 to $500 a year.
The ability to control a vehicle is strongly influenced by tire pressure. When the tire pressure is kept at proper levels, optimum vehicle braking, steering, handling and stability are accomplished. Low tire pressure can also lead to damage to both the tires and wheels.
A Michelin study revealed that the average driver doesn't recognize a low tire until it's 14 psi too low. One of the reasons is that today's radial tire is hard to judge visually because the sidewall flexes even when properly inflated.
Despite all the recent press about keeping tires properly inflated, new research shows that most drivers do not know the correct inflation pressure. In a recent survey, only 45 percent of respondents knew where to look to find the correct pressure, even though 78 percent thought they knew. Twenty-seven percent incorrectly believed the sidewall of the tire carries the correct information and did not know that the sidewall only indicates the maximum pressure for the tire, not the optimum pressure for the vehicle. In another survey, about 60% of the respondents reported that they check tire pressure but only before going on a long trip. The National Highway Traffic Safety Administration estimates that at least one out of every five tires is not properly inflated.
The problem is exacerbated with the new run-flat tires where a driver may not be aware that a tire is flat until it is destroyed. Run-flat tires can be operated at air pressures below normal for a limited distance and at a restricted speed (125 miles at a maximum of 55 mph). The driver must therefore be warned of changes in the condition of the tires so that she can adapt her driving to the changed conditions.
One solution to this problem is to continuously monitor the pressure and perhaps the temperature in the tire. Pressure loss can be automatically detected in two ways: by directly measuring air pressure within the tire or by indirect tire rotation methods. Various indirect methods are based on the number of revolutions each tire makes over an extended period of time through the ABS system and others are based on monitoring the frequency changes in the sound emitted by the tire. In the direct detection case, a sensor is mounted into each wheel or tire assembly, each with its own identity. An on-board computer collects the signals, processes and displays the data and triggers a warning signal in the case of pressure loss.
Under-inflation isn't the only cause of sudden tire failure. A variety of mechanical problems including a bad wheel bearing or a “dragging” brake can cause the tire to heat up and fail. In addition, as may have been a contributing factor in the Firestone case, substandard materials can lead to intra-tire friction and a buildup of heat. The use of re-capped truck tires is another example of heat caused failure as a result by intra-tire friction. An overheated tire can fail suddenly without warning.
The Transportation Recall Enhancement, Accountability, and Documentation Act, (H.R. 5164, or Public Law No. 106-414) known as the TREAD Act, was signed by President Clinton on Nov. 1, 2000. Section 12, TIRE PRESSURE WARNING, states that: “Not later than one year after the date of enactment of this Act, the Secretary of Transportation, acting through the National Highway Traffic Safety Administration, shall complete a rulemaking for a regulation to require a warning system in a motor vehicle to indicate to the operator when a tire is significantly under-inflated. Such requirement shall become effective not later than 2 years after the date of the completion of such rulemaking.” Thus, it is expected that a rule requiring continuous tire monitoring will take effect for the 2004 model year.
This law will dominate the first generation of such systems as automobile manufacturers move to satisfy the requirement. In subsequent years, more sophisticated systems that in addition to pressure will monitor temperature, tire footprint, wear, vibration, etc. Although the Act requires that the tire pressure be monitored, it is believed by the inventors that other parameters are as important as the tire pressure or even more important than the tire pressure as described in more detail below.
Consumers are also in favor of tire monitors. Johnson Controls' market research showed that about 80 percent of consumers believe a low tire pressure warning system is an important or extremely important vehicle feature. Thus, as with other safety products such as airbags, competition to meet customer demands will soon drive this market.
Although, as with most other safety products, the initial introductions will be in the United States, speed limits in the United States and Canada are sufficiently low that tire pressure is not as critical an issue as in Europe, for example, where the drivers often drive much faster.
The advent of microelectromechanical (MEMS) pressure sensors, especially those based on surface acoustical wave (SAW) technology, has now made the wireless and powerless monitoring of tire pressure feasible. This is the basis of the tire pressure monitors described below. According to a Frost and Sullivan report on the U.S. Micromechanical Systems (MEMS) market (June 1997): “A MEMS tire pressure sensor represents one of the most profound opportunities for MEMS in the automotive sector.”
There are many wireless tire temperature and pressure monitoring systems disclosed in the prior art patents such as for example, U.S. Pat. Nos. 4,295,102, 4,296,347, 4,317,372, 4,534,223, 5,289,160, 5,612,671, 5,661,651, 5,853,020 and 5,987,980 and International Publication No. WO 01/07271(A1), all of which are illustrative of the state of the art of tire monitoring and are incorporated by reference herein.
Devices for measuring the pressure and/or temperature within a vehicle tire directly can be categorized as those containing electronic circuits and a power supply within the tire, those which contain electronic circuits and derive the power to operate these circuits either inductively, from a generator or through radio frequency radiation, and those that do not contain electronic circuits and receive their operating power only from received radio frequency radiation. For the reasons discussed above, the discussion herein is mainly concerned with the latter category. This category contains devices that operate on the principles of surface acoustic waves (SAW) and the disclosure below is concerned primarily with such SAW devices.
International Publication No. WO 01/07271 describes a tire pressure sensor that replaces the valve and valve stem in a tire.
U.S. Pat. No. 5,231,827 contains a good description and background of the tire-monitoring problem. The device disclosed, however, contains a battery and electronics and is not a SAW device. Similarly, the device described in U.S. Pat. No. 5,285,189 contains a battery as do the devices described in U.S. Pat. Nos. 5,335,540 and 5,559,484. U.S. Pat. No. 5,945,908 applies to a stationary tire monitoring system and does not use SAW devices.
U.S. Pat. No. 5,987,980 describes a tire valve assembly using a SAW pressure transducer in conjunction with a sealed cavity. This patent does disclose wireless transmission. The assembly includes a power supply and thus this also distinguishes it from a preferred system of this invention. It is not a SAW system and thus the antenna for interrogating the device in this design must be within one meter, which is closer than needed for a preferred device of this invention.
U.S. Pat. No. 5,698,786 relates to the sensors and is primarily concerned with the design of electronic circuits in an interrogator. U.S. Pat. No. 5,700,952 also describes circuitry for use in the interrogator to be used with SAW devices. In neither of these patents is the concept of using a SAW device in a wireless tire pressure monitoring system described. These patents also do not describe including an identification code with the temperature and/or pressure measurements in the sensors and devices.
U.S. Pat. No. 5,804,729 describes circuitry for use with an interrogator in order to obtain more precise measurements of the changes in the delay caused by the physical or chemical property being measured by the SAW device. Similar comments apply to U.S. Pat. No. 5,831,167. Other related prior art includes U.S. Pat. No. 4,895,017.
Other patents disclose the placement of an electronic device in the sidewall or opposite the tread of a tire but they do not disclose either an accelerometer or a surface acoustic wave device. In most cases, the disclosed system has a battery and electronic circuits.
The following additional U.S. patents provide relevant information to this invention, and to the extent necessary, all of them are incorporated by reference herein: U.S. Pat. Nos. 4,361,026, 4,620,191, 4,703,327, 4,724,443, 4,725,841, 4,734,698, 5,691,698, 5,841,214, 6,060,815, 6,107,910, 6,114,971, 6,144,332.
U.S. Pat. No. 5,228,337 to Sharpe, et al. describes tire pressure and temperature measurement system in which the vehicle wheel tire inflation pressure is measured in real time by a sensor assembly mounted on a rotary part of the wheel. The assembly includes a piezoresistive cell exposed to inflation gas pressure and an electronics module comprising an assembly of three printed circuit boards (PCB). A power signal transmitted from the vehicle to the electronics module via a rotary transformer is conditioned by PCB to provide an energizing signal for the cell. Pressure and temperature signals output by the cell are received by the PCB and converted to digital form before being applied to address locations in a look-up table of PCB which holds pre-calibrated cell outputs. Data from the look-up table is processed to obtain a corrected real time pressure value which is transmitted to the vehicle. If desired, a temperature value may also be transmitted.
U.S. Pat. Nos. 5,600,301 and 5,838,229 to Robinson, III describe a remote tire pressure monitoring system employing coded tire identification and radio frequency transmission, and enabling recalibration upon tire rotation or replacement. The system indicates low tire pressure in vehicles, in which each vehicle wheel has a transmitter with a unique code, i.e., the transmitter is internal of the tire. A central receiver in the vehicle is taught, at manufacture, to recognize the codes for the respective transmitters for the vehicle, and also a common transmitter code, in the event one of the transmitters needs to be replaced. During vehicle operation and maintenance, when the tires are rotated, the system can be recalibrated to relearn the locations of the transmitters. The transmitters employ surface acoustic wave devices. An application specific integrated circuit encoder in each transmitter is programmed at manufacture, in accordance with its unique code, to send its information at different intervals, to avoid clash between two or more transmitters on the vehicle. The transmitters are powered by long-life batteries.
U.S. Pat. No. 5,880,363 to Meyer, et al. describes a method for checking air pressure in vehicle wheel tires wherein a pressure signal characteristic for the air pressure in the tire is picked up as a measured signal by a measurement device located in or on the tire of each motor vehicle wheel. A data signal containing a measured air pressure value derived from the pressure signal as well as an identification value characteristic for the respective transmitter device is generated and output by a transmitter device located in or on the tire of each motor vehicle wheel. The data signal output by the transmitter devices will be received by a reception device located at a distance to the motor vehicle wheels. The identification value of the transmitter device contained in the data signal will be compared by a control unit to identification comparison values assigned to the respective transmitter devices such that further processing of the data signal by the control unit will be effected only, if the identification value and the identification comparison value meet a specified assignment criterion. A drawback of this device is that it also uses a battery.
U.S. Pat. No. 5,939,977 to Monson describes a method and apparatus for remotely measuring the pressure and temperature of the gas in a vehicle wheel. The vehicle includes a frame member, a vehicle wheel mounted for rotation relative to the frame member about a rotation axis, and a modulator mounted on the vehicle wheel for movement therewith. The modulator generates a carrier signal including a first component encoding a plurality of consecutive data signals corresponding to a physical characteristic of the vehicle wheel, and the carrier signal including a second component identifying a portion of the respective one of the data signals
U.S. Pat. No. 5,963,128 to McClelland describes a remote tire pressure monitoring system which monitors a vehicle's tire pressures and displays real-time pressure values on a dashboard display while the vehicle is on the road. An electronic unit with pressure sensor, roll switch, reed switch, tilt switch, battery and control electronic, mounted to the valve stem inside each tire uses the pressure sensor to periodically measure the tire pressure, and uses a transmitter to transmit the measured pressure values, via RF transmission, to a dashboard mounted receiver. The receiver controls a display which indicates to the driver the real-time tire pressure in each wheel. The display also indicates an alarm condition when the tire pressure falls below certain predefined thresholds. The pressure values are compensated for temperature changes inside the tire, and also may be compensated for altitude changes.
U.S. Pat. No. 6,005,480 to Banzhof, et al. describes a snap-in tire valve including a valve body surrounded in part by a resilient element that forms an annular sealing surface configured to snap in place into a valve opening of a wheel. A tire pressure radio-frequency sending unit is mounted to the valve body, and a column extends from the sending unit. The region between the resilient element and the pressure sending unit defines an expansion volume that receives displaced portions of the resilient element during snap-in insertion of the valve body into a wheel opening, thereby facilitating insertion. Preferably the column defines a central passageway to facilitate insertion using standard insertion tools. In one version, two batteries are included in the sending unit, disposed on opposite sides of the column.
U.S. Pat. No. 6,034,597 to Normann, et al. describes a method for processing signals of a tire pressure monitoring system on vehicles in which a transmitter is mounted on each wheel of the vehicle and a reception antenna allocated to each transmitter is connected to the input of a common receiver. The transmitters transmit, at time intervals, data telegrams which contain an individual identifier and a data portion following the latter. The signals received simultaneously from the reception antennas and having the same identifier are conveyed in summed fashion to the receiver in a set manner.
U.S. Pat. No. 6,043,738 to Stewart, et al. describes a remote tire pressure monitoring system includes a sending unit for each monitored tire, and the sending units transmit RF signals, each including an identifier and a pressure indicator. A receiver operates in a learn mode in which the receiver associates specific identifiers either with the vehicle or with specific tires. During the learn mode the vehicle is driven at a speed above a threshold speed, such as thirty miles an hour, and identifiers are associated with either the vehicle or the respective tires of the vehicle only if they persist for a selected number of signals or frames during the learning period. In one example, the tires are inflated with different pressures according to a predetermined pattern, and the pressure indicators of the receive signals are used to associate individual tire positions with the respective sending units.
U.S. Pat. No. 6,046,672 to Pearman describes a tire condition indicating device having a detector for detecting the condition of a tire on a wheel of a vehicle rotatable about a wheel axis, preferably for detecting pressure of the tire. A signal emitter emits a signal when the detector detects the condition and a power supply device provides power to the signal emitter. The power supply device has an electric power generator including first and second parts that are relatively rotatable about a generator axis, the first part connected to the wheel to rotate.
U.S. Pat. No. 6,053,038 to Schramm, et al. describes an internal-to-vehicle mechanism for monitoring the air pressure of a tire of a vehicle. The mechanism includes a sensor, detecting the tire pressure, which rotates, together with an electrotechnical first device, synchronously with the wheel and which, as a function of the tire air pressure that is determined, modifies parameters of the first device, namely the energy uptake of the first device. A stationary electrotechnical second device radiates an electric and/or magnetic, in particular electromagnetic, field through which the first device passes at, preferably, each wheel rotation with an uptake of energy from the field. A monitoring device detects the energy uptake and/or energy release of the second device.
U.S. Pat. No. 6,101,870 to Kato, et al. describes a device for monitoring the air pressure of a wheel. The device prevents a decrease in the transmission level of radio waves caused by impedance mismatch between an antenna, which radiates the radio waves, and a circuit, which produces signals that are to be radiated as the radio waves. The device includes a valve stem through which air is charged. The valve stem extends through a vehicle wheel. A transmitter is secured to the wheel to transmit a signal representing the air pressure of the wheel to a receiver installed in the vehicle. The device further includes a case attached to the wheel. The case is connected to the valve stem. An electric circuit is accommodated in the case to detect the air pressure and convert the detected pressure to an electric signal. An antenna radiates the signal produced by the electric circuit and is arranged about the valve stem. A conveying mechanism conveys the signals produced by the electric circuit to the antenna.
U.S. Pat. No. 6,112,585 to Schrottle, et al. describes a tire pressure monitoring device for a vehicle having several wheels comprises a central receiving and evaluation device at the vehicle. A receiving antenna is arranged stationarily at the vehicle structure adjacent to at least each active wheel and thus attributed to that specific wheel. All receiving antennas are connected via a distinctive connecting line with a single receiver means. The receiver means comprises a multiplexer-circuit connecting per time interval only one single selected receiving antenna or several selected receiving antennas with the receiving means. Further, the receiver means sense a field strength of each specific radiogram and thus select the specific receiving antenna comprising the highest field strength of a received radiogram during the specific time interval. Thus, central evaluation means may attribute a specific radiogram to the specific wheel arranged adjacent to the receiving antenna comprising the highest field strength of a received radiogram during the specific time interval.
None of these patents show a temperature sensor mounted entirely at a location external of and apart from the tire and coupling the temperature sensor with a unit capable of receiving power either inductively or through radio frequency energy transfer in order to enable the temperature sensor to conduct a temperature measurement. Rather, all of the tire monitoring systems entail the use of a sensor or other device mounted on the tire or formed in connection with the tire.
1.5 Occupant Sensing
It is now generally recognized that it is important to monitor the occupancy of a passenger compartment of a vehicle. For example see U.S. Pat. Nos. 5,653,462, 5,694,320, 5,822,707, 5,829,782, 5,835,613, 5,485,000, 5,488,802, 5,901,978, 5,943,295, 6,309,139, 6,078,854, 6,081,757, 6,088,640, 6,116,639, 6,134,492, 6,141,432, 6,168,198, 6,186,537, 6,234,519, 6,234,520, 6,242,701, 6,253,134, 6,254,127, 6,270,116, 6,279,946, 6,283,503, 6,324,453, 6,325,414, 6,330,501, 6,331,014, RE37,260, 6,393,133, 6,397,136, 6,412,813, 6,422,595, 6,452,870, 6,442,504, 6,445,988, 6,442,465 (Breed et al.) which describe several vehicle interior monitoring systems that utilize pattern recognition techniques and wave-receiving sensors to obtain information about the occupancy of the passenger compartment and uses this information to affect the operation of one or more systems in the vehicle, including an occupant restraint device, an entertainment system, a heating and air-conditioning system, a vehicle communication system, a distress notification system, a light filtering system and a security system.
Of particular interest, Breed et al. mentions that the presence of a child in a rear facing child seat placed on the right front passenger seat may be detected as this has become an industry-wide concern to prevent deployment of an occupant restraint device in these situations. The U.S. automobile industry is continually searching for an easy, economical solution, which will prevent the deployment of the passenger side airbag if a rear facing child seat is present.
1.6 Vehicle or Component Control
Based on the monitoring of vehicular components, systems and subsystems as well as to the measurement of physical and chemical characteristics relating to the vehicle or its components, systems and subsystems, it becomes possible to control and/or affect one or more component, vehicular system or the vehicle itself as discussed below.
2.0 Telematics
Every automobile driver fears that his or her vehicle will break down at some unfortunate time, e.g., when he or she is traveling at night, during rush hour, or on a long trip away from home. To help alleviate that fear, certain luxury automobile manufacturers provide roadside service in the event of a breakdown. Nevertheless, unless the vehicle is equipped with OnStar® or an equivalent service, the vehicle driver must still be able to get to a telephone to call for service. It is also a fact that many people purchase a new automobile out of fear of a breakdown with their current vehicle. The inventions described herein are primarily concerned with preventing breakdowns and with minimizing maintenance costs by predicting component failure that would lead to such a breakdown before it occurs.
Another important aspect disclosed in the Breed et al. patents relates to the operation of the cellular communications system in conjunction with the vehicle interior monitoring system. Vehicles can be provided with a standard cellular phone as well as the Global Positioning System (GPS), an automobile navigation or location system with an optional connection to a manned assistance facility. In the event of an accident, the phone may automatically call 911 for emergency assistance and report the exact position of the vehicle. If the vehicle also has a system as described below for monitoring each seat location, the number and perhaps the condition of the occupants could also be reported. In that way, the emergency service (EMS) would know what equipment and how many ambulances to send to the accident site. Moreover, a communication channel can be opened between the vehicle and a monitoring facility/emergency response facility or personnel to determine how badly people are injured, the number of occupants in the vehicle, and to enable directions to be provided to the occupant(s) of the vehicle to assist in any necessary first aid prior to arrival of the emergency assistance personnel.
Communications between a vehicle and a remote assistance facility are also important for the purpose of diagnosing problems with the vehicle and forecasting problems with the vehicle, called prognostics. Motor vehicles contain complex mechanical systems that are monitored and regulated by computer systems such as electronic control units (ECUs) and the like. Such ECUs monitor various components of the vehicle including engine performance, carburation, speed/acceleration control, transmission, exhaust gas recirculation (EGR), braking systems, etc. However, vehicles perform such monitoring typically only for the vehicle driver and without communication of any impending results, problems and/or vehicle malfunction to a remote site for trouble-shooting, diagnosis or tracking for data mining.
In the past, systems that provide for remote monitoring did not provide for automated analysis and communication of problems or potential problems and recommendations to the driver. As a result, the vehicle driver or user is often left stranded, or irreparable damage occurs to the vehicle as a result of neglect or driving the vehicle without the user knowing the vehicle is malfunctioning until it is too late, such as low oil level and a malfunctioning warning light, fan belt about to fail, failing radiator hose etc.
U.S. Pat. No. 5,400,018 (Scholl et al.) describes a system for relaying raw sensor output from an off road work site relating to the status of a vehicle to a remote location over a communications data link. The information consists of fault codes generated by sensors and electronic control modules indicating that a failure has occurred rather than forecasting a failure. The vehicle does not include a system for performing diagnosis. Rather, the raw sensor data is processed at an off-vehicle location in order to arrive at a diagnosis of the vehicle's operating condition. Bi-directional communications are described in that a request for additional information can be sent to the vehicle from the remote location with the vehicle responding and providing the requested information but no such communication takes place with the vehicle operator and not of an operator of a vehicle traveling on a road. Also, Scholl et al. does not teach the diagnostics of the problem or potential problem on the vehicle itself nor does it teach the automatic diagnostics or any prognostics. In Scholl et al. the determination of the problem occurs at the remote site by human technicians.
U.S. Pat. No. 5,955,942 (Slifkin et al.) describes a method for monitoring events in vehicles in which electrical outputs representative of events in the vehicle are produced, the characteristics of one event are compared with the characteristics of other events accumulated over a given period of time and departures or variations of a given extent from the other characteristics are determined as an indication of a significant event. A warning is sent in response to the indication, including the position of the vehicle as determined by a global positioning system on the vehicle. For example, for use with a railroad car, a microprocessor responds to outputs of an accelerometer by comparing acceleration characteristics of one impact with accumulated acceleration characteristics of other impacts and determines departures of a given magnitude from the other characteristics as a failure indication which gives rise of a warning.
Of course there are many areas of the country where cell phone reception is not available and thus a system that relies on the availability of such a system for diagnostics will not always be available and thus has a significant failure mode. Furthermore, it would be difficult if not impossible for such a location to have all of the information to diagnose problems with all vehicle models that are on the road and to be able to retrieve that information and act on raw data on a continuous basis to keep track of whether all vehicles on the roadways are operating properly and to forecast all potential problems with each vehicle. Thus, this function must be resident on the vehicle. Additionally is a human operator is required then the system quickly becomes unmanageable.
3.0 Wiring and Busses
It is not uncommon for an automotive vehicle today to have many motors, other actuators, lights etc., controlled by one hundred or more switches and fifty or more relays and connected together by almost five hundred meters of wire, and close to one thousand pin connections grouped in various numbers into connectors. It is not surprising therefore that the electrical system in a vehicle is by far the most unreliable system of the vehicle and the probable cause of most warranty repairs.
Unfortunately, the automobile industry is taking a piecemeal approach to solving this problem when a revolutionary approach is called for. Indeed, the current trend in the automotive industry is to group several devices of the vehicle's electrical system together which are located geometrically or physically in the same area of the vehicle and connect them to a zone module which is then connected by communication and power buses to the remainder of the vehicle's electrical system. The resulting hybrid systems still contain substantially the same number and assortment of connectors with only about a 20% reduction in the amount of wire in the vehicle.
4.0 Displays and Inputs to Displays
In an existing heads up display, information is projected onto a specially treated portion of the windshield and reflected into the eyes of the driver. An important component of a head-up display system is known as the combiner. The combiner is positioned forward of the driver and extends partly across his or her view of the real world scene. It is usually either on the interior surface of or laminated inside of the windshield. It is constructed to permit light from the real world scene ahead of the vehicle to pass through the combiner and to reflect light information of one or more particular wavelengths propagating from a source within the vehicle. The information is projected onto the combiner using suitable optical elements. The light rays reflected by the combiner are typically collimated to present an image of the information focused at optical infinity permitting the driver to simultaneously view the real world scene and the displayed information without changing eye focus.
Some combiners are simply semi-reflecting mirrors while a particularly effective combiner can be constructed using a hologram or a holographic optical element. In a currently used heads up display in motor vehicles, the motorist views the forward outside real world scene through the windshield. Information pertaining to the operational status of the vehicle is displayed on a heads up display system providing vehicle information, such as fuel supply and vehicle speed, positioned within the motorist's field of view through the windshield thereby permitting the motorist to safely maintain eye contact with the real world scene while simultaneously viewing the display of information. However, such heads up displays are not interactive.
Heads up displays are widely used on airplanes particularly military airplanes. Although many attempts have been made to apply this technology to automobiles, as yet few heads up display systems are on production vehicles. Nevertheless, it is anticipated that more such systems will appear on vehicles in the near future. One reason that heads up displays have not been widely implemented is that vehicle operators have not been willing to pay the cost of such a system merely to permit the operator to visualize his speed or the vehicle temperature, for example, without momentarily taking his eyes from the road. In other words, the service provided by such systems is not perceived to be worth the cost.
There are functions other than viewing the vehicle gages that a driver typically performs that require significantly more attention than a momentary glance at the speedometer. Such functions have heretofore not been considered for the heads up display system. These functions are primarily those functions that are only occasionally performed by the vehicle operator and yet require significant attention. As a result, the vehicle operator must remove his eyes from the road for a significant time period while he performs these other functions creating a potential safety problem. One example of such a function is the adjustment of the vehicle entertainment system. The vehicle entertainment system has become very complex in modern automobiles and it is now very difficult for a vehicle driver to adjust the system for optimum listening pleasure while safely operating the vehicle.
Other similar functions include the adjustment of the heating, ventilation, air conditioning and defrosting system, the dialing and answering of cellular phone calls, as well as other functions which are contemplated for future vehicles such as navigational assistance, Internet access, in-vehicle messaging systems, traffic congestion alerts, weather alerts, etc. Each of these functions, if performed by a driver while operating the vehicle, especially under stressful situations such as driving on congestion highways or in bad weather, contribute an unnecessary risk to the driving process. While a driver is attempting to operate the vehicle in a safe manner, he or she should not be required to remove his or her eyes from the road in order to adjust the radio or make a phone call. Therefore, one purpose of this invention is to minimize this risky behavior by permitting the operator to perform these functions without taking his or her eyes off of the road. As discussed in greater detail below, this is accomplished through the use of a heads up display system combined with a touch pad located, for example, at the center of the steering wheel within easy reach of the driver, a gesture recognition input system, or a voice input system.
4.1 Prior Art Related to Heads Up Display Systems
There are many patents and much literature that describe the prior art of heads up displays. Among the most significant of the patents are:
U.S. Pat. No. 4,218,111 which describes a lens system for one of the early holographic heads up display units.
U.S. Pat. No. 4,309,070 which describes an aircraft head up display system for pilots.
U.S. Pat. No. 4,613,200 which describes a system for using narrow wavelength bands for the heads up display system. It describes a rather complicated system wherein two sources of information are combined. This patent is believed to be the first patent teaching a heads up display for automobiles.
U.S. Pat. No. 4,711,544 which describes a head up display for an automobile and clearly describes the process by which the focal length of the display is projected out front of the automobile windshield. In this manner, the driver does not have to focus on a display which is close by as, for example, on the instrument panel. Thus, the driver can continue to focus on the road and other traffic while still seeing the heads up display.
U.S. Pat. No. 4,763,990 which describes a method for reducing flare or multiple images resulting in a substantially aberration free display. This is a problem also discussed by several of the other prior art patents.
U.S. Pat. No. 4,787,040 which describes another type display system for automobiles which is not a heads up display. This patent shows the use of “an infrared touch panel or Mylar™ touch switch matrix mounted over the face of the display”.
U.S. Pat. No. 4,787,711 which describes and solves problems of double reflection or binocular parallax that results from conventional heads up displays for use in automobiles.
U.S. Pat. No. 4,790,613 which presents a low-cost heads up display with fixed indicia. The message is fixed but displayed only as needed.
U.S. Pat. No. 4,886,328 which shows a heads up display device and describes a method for preventing damage to the optics of the system caused by sunlight.
U.S. Pat. No. 4,973,132 which describes a polarized holographic heads up display which provides for increased reflectivity and image contrast.
U.S. Pat. No. 5,013,135 which describes a heads up display using Fresnel lenses to reduce the space required for installation of the system.
U.S. Pat. No. 5,157,549 which describes another method of reducing the damage to the heads up display optics by restricting the wavelengths of external light which are reflected into the heads up display optics.
U.S. Pat. No. 5,210,624 which describes a heads up display wherein all light from the environment is allowed to pass through the combiner except light having a frequency equal to the frequency generated by the heads up display. The alleged improvement is to also filter out light from the environment that is of a complementary color to the light from the heads up display.
U.S. Pat. No. 5,212,471 which describes a method for reducing the reflections from the outside windshield surface which produces ghost images.
U.S. Pat. No. 5,229,754 which describes apparatus for increasing the path length of the heads up display using a reflecting plate. This improves the quality of the heads up display while maintaining a compact apparatus design. This added travel of the light rays is needed since in this system the virtual image is located as far in front of the vehicle windshield as the distance from the information source to the heads up display reflector.
U.S. Pat. No. 5,231,379 which describes a method for compensating for the complex aspheric curvature of common windshields. It also provides means of adjusting the vertical location of the reflection off the windshield to suit the size of a particular driver or his preferences.
U.S. Pat. No. 5,243,448 which describes a low-cost heads up display for automobiles.
U.S. Pat. No. 5,289,315 which describes apparatus for displaying a multicolored heads up display. The technique uses two films having different spectral reflectivities.
U.S. Pat. No. 5,313,292 which describes a method for manufacturing a windshield containing a holographic element. This patent presents a good description of a heads-up display unit including mechanisms for reducing the heat load on the LCD array caused by the projection lamp and means for automatically adjusting the intensity of the heads up display so that the contrast ratio between the heads up display and the real world is maintained as a constant.
U.S. Pat. No. 5,313,326 which describes a heads up display and various methods of improving the view to drivers looking at the heads up display from different vertical and lateral positions. The inventor points out that “ . . . the affective eye box presented to the driver, i.e. the area within which he will be able to see the image is inherently limited by the effective aperture of the optical projection unit”.
The inventor goes on to teach that the eye box should be as large as possible to permit the greatest tolerance of the system to driver height variation, driver head movement, etc. It is also desirable to have a compact optical projection system as possible since available space in the car is limited. There are, however, limitations on the length of the projection unit and the size of the eye box that is achievable.
While the use of more powerful optics will permit a shorter physical length unit for a fixed image projection distance, this will give a higher display magnification. The higher the magnification, the smaller the actual display source for a specific image size. Display resolution then becomes a critical factor. A second limitation of optical systems is that for a given eye box a shorter focal length system cannot achieve as good an image quality as a long focal length system.
U.S. Pat. No. 5,329,272, as well as many of the other patents cited above, which describes the use of a heads up display to allow the operator or driver to watch the speedometer, revolution counter, directional indicators, etc. while keeping his or her eyes on the road. This patent is concerned with applying or adapting a large bulky optical system to the vehicle and solves problem by placing the main elements of this optical system in a direction parallel to the transverse axis of the vehicle. This patent also describes a method for adjusting the heads up display based on the height of the driver. It mentions that using the teachings therein that the size of the driver's binocular or eye box is 13 cm horizontal by 7 cm vertical.
U.S. Pat. No. 5,379,132 which attempts to solve the problem of the limited viewing area provided to a driver due to the fact that the size of the driver is not known. A primary object of the invention is to provide a display having an enlarged region of observation. This is done by reducing the image so that more information can be displayed on the heads up display.
U.S. Pat. No. 5,414,439 which states that such heads up displays have been quite small relative to the roadway scene due to the limited space available for the required image source and projection mirrors.
U.S. Pat. No. 5,422,812 which describes an in route vehicle guidance system using a heads up display, but not one that is interactive.
U.S. Pat. No. 5,486,840 which describes a heads up display which purportedly eliminate the effect where sunlight or street lights travel down the path of the heads up display optics and illuminate the projection surface and thereby cause false readings on the heads up display. This problem is solved by using circularly polarized light.
U.S. Pat. No. 5,473,466 describes a miniature high resolution display system for use with heads up displays for installation into the helmets of fighter pilots. This system, which is based on a thin garnet crystal, requires very little power and maintains a particular display until display is changed. Thus, for example, if there is a loss of power the display will retain the image that was last displayed. This technology has the capability of producing a very small heads up display unit as will be described more detail below.
U.S. Pat. No. 5,812,332 which describes a windshield for a head up display system that reduces the degree of double imaging that occurs when a laminated windshield is used as the combiner in the display system.
U.S. Pat. No. 5,859,714 which describes a method for making the combiner such that a colored heads up display can be created.
Finally, U.S. Pat. No. 5,724,189 which describes methods and apparatus for creating aspheric optical elements for use in a heads up display.
4.2 Summary of the Heads Up Prior Art
All of the heads up display units described are for providing an alternate to viewing the gages on the instrument panel or at most the displaying of a map. That is, all are passive systems. Nowhere has it been suggested in the above-mentioned prior art to use the heads up display as a computer screen for interactive use by the vehicle operator where the driver can operate a cursor and/or otherwise interact with the display.
No mention is made in the above-mentioned prior art of the use of a heads up display for: the Internet; making or receiving phone calls; compartment temperature control; control of the entertainment system; active route guidance with input from an external source such as OnStar™; in vehicle signage; safety alerts; weather alerts; traffic and congestion alerts; video conferencing; TV news broadcasts; display of headlines, sports scores or stock market displays; or of switches that can be activated orally, by gesture or by a touch pad in the center of the steering wheel or elsewhere.
Furthermore, there does not appear to be any examples of where a heads up display is used for more than one purpose, that is, where a variety of different pre-selectable images are displayed.
4.3 Background on Touch Pad Technologies
Touch pads are closely related to their “cousins”, touch screens. Both use the operator's fingers as the direct link between the operator and the computer. In some cases, a stylus is used but not for the cases to be considered here. In simple cases, touch pads can be used to operate virtual switches and, in more complicated cases, the movement of the operators finger controls a cursor, which can be used to select from a range of very simple to very complicated functions. Several technologies have evolved which will now be described along with some of their advantages and shortcomings.
Capacitive touch pads use the electrical (conductive and dielectric) properties of the user's finger as it makes contact with the surface of the pad. This capacitive technology provides fast response time, durability and a tolerance for contamination. Generally, grease, water and dirt will not interfere with the operation of the capacitive touch pad. Unfortunately, this technology will not work well for steering wheel applications, for example, when the driver is wearing gloves.
Projected capacitive touch pads sense changes in the electrical field adjacent the touch pad. This technology will work with a driver wearing gloves but does not have as high a resolution as the standard capacitive touch pads.
Infrared touch pads contain a grid of light beams across the surface of the pad and check for interruptions in that grid. This system is somewhat sensitive to contamination that can block the transmitters or receivers.
Surface acoustic wave (SAW) touch pads send sound waves across the surface of the touch pad and look for interruptions or damping caused by the operator's fingers. This technology requires the use of a rigid substrate such as glass that could interfere with the operation of the airbag deployment door. It is also affected by contaminants which can also absorb the waves.
Guided acoustic wave technology is similar to SAW except that it sends the waves through the touch pad substrate rather than across the surface. This technology also requires a rigid substrate such as glass. It is additionally affected by contamination such as water condensation.
Force sensing touch pads measure the actual force placed on the pad and is measured where the pad is attached. Typically, strain gages or other force measuring devices are placed in the corners of a rigid pad. This technology is very robust and would be quite applicable to steering wheel type applications, however, it generally has less resolution than the other systems. Force sensing touch pads are either strain gage or platform types. The strain gage touch pad measures the stresses at each corner that a touch to the pad creates. The ratio of the four readings indicates the touch point coordinates. The platform touch pad instead rests on a platform with force measurement sensors at the supports. A touch onto the touch pad translates to forces at the supports.
Resistive touch pads use a flexible resistive membrane, a grid of insulators and a secondary conducting pad to locate the touch point. This pad generally has higher resolution than the force sensing touch pads and is equally applicable to steering wheel type applications. A further advantage is that it can be quite thin and does not generally require a rigid substrate which can interfere with the deployment of the airbag door. Resistive technology touch screens are used in more applications than any other because of the high accuracy fast response and trouble-free performance in a variety of harsh applications.
There are many U.S. patents and other publications that describe touch pad technologies primarily as they relate to inputting data into a computer. Among the significant patents are:
U.S. Pat. No. 4,190,785 which describes a touch pad using a piezoelectric layer. When a finger pressure is placed on the piezoelectric, a voltage is generated. The touch pad actually consists of an array of sensors rather than a continuously varying sensing element. One advantage of the system is that it can be passive. The piezoelectric coating is approximately 0.005 inches thick.
U.S. Pat. No. 4,198,539 which describes a touch pad based on resistance. Through a novel choice of resistors and uniform resistive pad properties, the inventor is able to achieve a uniform electric field in the resistance layer of the touch pad.
U.S. Pat. No. 4,328,441 which describes a “piezoelectric polymer pressure sensor that can be used to form a pressure sensitive matrix keyboard having a plurality of keyboard switch positions arranged in a plurality of rows and columns”. The piezoelectric electric polymer film is made from polyvinylidene fluoride. This is only one example of the use of the piezoelectric polymer and some others are referenced in this patent. This touch pad is set up as a series of switches rather than a continuous function.
U.S. Pat. No. 4,448,837 which describes the use of a silicone rubber elastic sheet which has been partially filled with conductive particles of various sizes as part of a resistive touch pad.
U.S. Pat. No. 4,476,463 which describes a touch pad system for use as an overlay on a display that can detect and locate a touch at any location anywhere on the display screen. In other words, it is a continuously variable system. This system is based on a capacitive system using an electrically conductive film overlaying the display screen.
U.S. Pat. No. 4,484,179 which describes a touch sensitive device which is at least partially transparent to light. A flexible membrane is suspended over a CRT display and when pushed against the display it traps light emitted at the contact point by the scanning system. This trapped light can be sensed by edge mounted sensors and the position of the touch determined based on the known position of the scan when the light was detected.
U.S. Pat. No. 4,506,354 which describes an ultrasonic touch pad type device wherein two ultrasonic transducers transmit ultrasound through the air and receive echoes based on the position of a finger on the touch pad.
U.S. Pat. No. 4,516,112 which describes another implementation of a touch pad using a piezoelectric film.
U.S. Pat. No. 4,633,123 which describes another piezoelectric polymer touch screen, in this case used as a keyboard apparatus.
U.S. Pat. Nos. 4,745,301 and 4,765,930 which describe a deformable pressure sensitive electro-conductive switch using rubber which is loaded with conductive particles and which could be used in a touch switch or touch pad configuration.
U.S. Pat. No. 4,904,857 which describes a touch screen based on light emitting diodes (LEDs) and receptors wherein light beams are sent parallel to and across the top of the video screen and the interruption of these light beams is sensed.
U.S. Pat. No. 4,963,417 which describes a touch pad consisting of a conductive layer and a layer of deformable insulating particles and a conductive film layer. Pressure on the conductive film layer causes the insulating deformable particles to deform and permits contact between the conductive film and the conductive substrate that can be sensed by resistant measurements.
U.S. Pat. No. 4,964,302 which describes a tactile sensor which can be used by robots for example. The tactile sensor consists of a series of ultrasonic pads and a deformable top layer. When the deformable layer is compressed, the compression can be sensed by the time of flight of the ultrasonic waves by the ultrasonic sensor and therefore both the location of the compression can be determined and the amount compression or force. Such an arrangement is applicable to the touch pads of the current invention as described below. This permits an analog input to be used to control the radio volume, heating or air conditioning temperature, etc.
U.S. Pat. No. 5,008,497 which describes an accurate means for measuring the touch position and pressure on a resistive membrane.
U.S. Pat. No. 5,060,527 which is another example of the tactile sensor that is capable of measuring variable force or pressure. This patent uses an electrically conductive foam as the variable resistance that permits force to be measured.
U.S. Pat. No. 5,159,159 which is another example of a touch pad that is based on resistance and provides the X and Y position of the finger and the pressure at the touch point.
U.S. Pat. No. 5,164,714 which is another system using light emitters and detectors creating a field of light beams going across the surface of the touch pad in both X and Y directions.
U.S. Pat. No. 5,374,449 which describes a monolithic piezoelectric structural element for keyboards which can be used to form discrete switching elements on the pad.
U.S. Pat. No. 5,376,946 which describes a touch screen made of two transparent conductive members which when caused to contract each other change the resistance of the circuit such that, by alternately applying a voltage to the X and Y edges, the location of the touch point can be determined.
A capacitive based touch screen is illustrated in U.S. Pat. No. 5,386,219.
U.S. Pat. No. 5,398,962 which describes a horn activator for steering wheels with airbags. This horn activator switch can be made part of the touch pad as discussed below whereby when the pressure exceeds a certain amount, a horn blows rather than or in addition to activating the heads up display.
U.S. Pat. No. 5,404,443 which describes a CRT display with a touch pad overlay for use in an automobile.
U.S. Pat. No. 5,453,941 which describes a touch pad of the resistive type which also measures pressure as well as location of the touch. This patent uses two separate boards, one for the X coordinate and one for the Y coordinate. A pressure applied against the point located on the X coordinate resistance board causes the X coordinate resistance board to make contact with the Y coordinate resistance board at a point located on the Y coordinate resistance board. The contact is through a contact resistance the magnitude of which is inversely proportional to the pressure applied.
U.S. Pat. No. 5,518,078 which is another example were separate films are used for the X and Y direction. Voltages are selectively applied to the film for measuring the X coordinate and then to the film for measuring the Y coordinate. The pressure of the touch is determined by the contact resistance between the X and Y films.
Most of the prior art devices described above have an analog input, that is, the resistance or capacitance is continuously varying as the pressure point moves across the pad. U.S. Pat. No. 5,521,336, on the other hand, describes a touch pad which provides a digital input device by using sets of parallel strips in one layer orthogonal to another set of parallel strips in another layer. Upon depressing the surface, the particular strips which make contact are determined. These are known as high-density switch closure type touch pad sensors.
U.S. Pat. No. 5,541,372 which describes the use of strain gages to detect deformation of the touch panel itself as result of force being applied. Strain gages are physically integrated with the panel and measure the strain on the panel. An important feature of the invention of this patent is that it measures the deformation of panel itself instead of the deformation of the suspension members of the panel as in the prior art.
U.S. Pat. No. 5,541,570 which describes a force sensing ink that is used in U.S. Pat. No. 5,563,354 to form a thin film force sensors to be used, for example, for horn activation.
U.S. Pat. No. 5,673,041 which describes a reflective mode ultrasonic touch sensitive switch. A touch changes the reflectivity of a surface through which the ultrasound is traveling and changes the impedance of the transducer assembly. This switch can be multiplied to form a sort of digital touch pad. A piezoelectric polymer film is used presumably to maintain the transparency of the switch.
U.S. Pat. No. 5,673,066 which relates to a coordinate input device based on the position of a finger or pen to a personal computer. This patent provides various means for controlling the motion of a cursor based on the motion of a finger and also of providing a reliable switching function when an item has been selected with the cursor. The invention describes the use of touch pressure to indicate the speed with which the cursor should move. A light touch pressure provides for a rapid movement of cursor whereas a strong touch pressure signifies a slow movement. The pressure on the touch pad is determined using four piezoelectric elements for converting pressures to voltages that are arranged on the four corners of the back surface of the rigid plate.
U.S. Pat. No. 5,686,705 which describes a touch pad consisting of a conductive surface containing three electrodes, a compressible insulating layer and a top conductive layer such that when the top conductive layer is depressed it will receive signals from the three electrodes. These signals are transmitted in pairs thereby permitting the location of the contact point on a line bisecting the two electrodes, then by using another pair, a second line can be determined and the intersection of those two lines fixes the point. The determination is based on the level of signal that is inversely proportional to the resistance drop between the contact point in the transmission point.
U.S. Pat. No. 5,917,906 which describes an alternate input system with tactile feedback employing the use of snap domes arranged in the predetermined spaced apart arrangement.
U.S. Pat. No. 5,933,102 which describes an array of capacitive touch switches.
U.S. Pat. No. 5,942,733 which describes a capacitive touch pad sensor capable of being actuated with a stylus input. The consists of a plurality of first parallel conductive traces running in the X direction and a plurality of second parallel conductive traces running in the Y direction. A layer of pressure conductive material is disposed over one of the faces of the substrate which in turn is covered with a protective layer. As the conductive later is moved toward the arrays of substrates the capacitance between the conductive later and each of the substrates is changed which is measurable. A capacitive touch pad has the advantage that it requires much less force than a resistive touch pad. The traces are actually put on both sides of substrate with the X traces going one way and Y traces the other way. An alternative would be to use a flex circuit.
International Patent Publication No. WO98/43202 which describes a button wheel pointing device for use with notebook personal computers.
International Publication No. WO98/37506 reserves various parts of the touch pad for command bar or scroll bar functions.
U.S. Pat. No. 5,374,787 which describes a two-dimensional capacitive sensing system equipped with a separate set of drive and sense electronics for each row and column of the capacitive tablet. The device capacitively senses the presence of the finger and determines its location. This concept is further evolved in U.S. Pat. Nos. 5,841,078, 5,861,583, 5,914,465, 5,920,310 and 5,880,411. U.S. Pat. No. 5,841,078 makes use in one embodiment of a neutral network to interpret situations when more than one finger is placed on the touch pad. This allows the operator to use multiple fingers, coordinated gestures etc. for complex interactions. The traces can be placed on a printed circuit board or on a flex circuit. The sensor also measures finger pressure.
U.S. Pat. No. 5,861,583 which provides a two-dimensional capacitive sensing system that cancels out background capacitance effects due to environmental conditions such as moisture
Other capacitive prior art U.S. patents include U.S. Pat. Nos. 5,305,017, 5,339,213, 5,349,303 and 5,565,658. These patents also cover associated apparatus for capacitive touch pads sensors.
U.S. Pat. No. 5,565,658 which describes a system that can be used with gloves since the finger need not contact the surface of the touch pad and also describes a technique of making the touch pad using silk screening and a variety of inks, some conducting some non-conducting. The resulting array is both thin and flexible that allows it to be formed into curved surfaces such as required for a steering wheel mounted touch pad.
U.S. Pat. No. 5,940,065 which describes a mapping method of how to compensate for systematic and manufacturing errors which appear in a resistive touch sensor pad.
U.S. Pat. No. 5,694,150 which provides a graphical user interface system to permit multiple users of the same system. Such a system would be applicable when both the driver and passenger are viewing the same output on different heads up or other displays. This could also be useful, for example, when the passenger is acting as the navigator indicating to the driver on the heads up display where he is now and where he should go. Alternately, the navigator could be a remote access operator giving directions to the driver as to how to get to a specific location.
Touch pads that are commercially available include, for example, model TSM946 as supplied by Cirque Corporation and others supplied by the Elo and Synaptics corporations.
A human factors study has shown that the ideal size of the square target for the 95 percentile male population should be about 2.4 cm by 2.4 cm as reported in “A Touch Screen Comparison Study: Examination Of Target Size And Display Type On Accuracy And Response Time” by S. Gregory Michael and Michael E. Miller, Eastman Kodak Co. Rochester, N.Y.
4.4 Summary of the Touch Pad Prior Art
As can be appreciated from the sampling of touch pad patents and publications listed above, many technologies and many variations are available for touch pad technology. In particular, most of these designs are applicable for use, for example, as a touch pad mounted in the center of a steering wheel in accordance with the invention, as discussed more fully below. In general, the resolution required for a touch pad for a steering wheel application probably does not have to be as high as the resolution required for entering drawing or map data to a computer database, for example. A vehicle driver is not going to be able to focus intently on small features of the display. For many cases, a few switch choices is all that will be necessary. This would allow the driver to use the first screen to select among the major function groups that he or she is interested in, which might comprise the entertainment system, navigation system, Internet, telephone, instrument panel cluster, and perhaps one or two additional subsystems. Once he or she selects the system of interest by pressing a virtual button, he or she would then be presented with a new display screen with additional options. If the entertainment system had been chosen, for example, the next series of what choices would include radio, satellite radio, Internet radio, TV, CD, etc. Once the choice among these alternatives has been selected the new screen of button choices would appear.
For other more involved applications, actual control of cursor might be required in much the same way that a mouse is used to control the cursor on a personal computer. In fact, the heads up display coupled with the steering wheel mounted touch pad can in fact be a personal computer display and control device. The particular choice of system components including the heads up display technology and the touch pad technology will therefore depend on the sophistication of the particular system application and the resulting resolution required. Therefore, essentially all of the technologies described in the above referenced prior art touch pad patents are applicable to the invention to be described herein. Therefore, all of the above patents are included herein in their entirety by reference.
Generally, the steering wheel mounted touch pad and heads up display system will result in safer driving for the vehicle operator. This is because many functions that are now performed require the driver to take his or her eyes from the road and focus on some other control system within the vehicle. With the system of this invention, this will not be necessary. On the other hand, the potential exists for adding many more functions, some of which may become very distracting. It is envisioned, therefore, that implementation of the system will be in stages and to a large degree will be concomitant with the evolution of other safety systems such as autonomous vehicles. The first to be adopted systems will likely be relatively simple with low resolution screens and minimum choices per screen. Eventually, full-length movies may someday appear on the heads up display for the entertainment of the vehicle operator while his vehicle is being autonomously guided.
The preferred touch pad technologies of those listed above include capacitance and resistance technologies. Most of the capacitance technologies described require the conductivity of the operator's finger and therefore will not functions if the driver is wearing gloves. Some of the patents have addressed this issue and with some loss of resolution, the standard capacitive systems can be modified to sense through thin driving gloves. For thicker gloves, the projected capacitive systems become necessary with an additional loss of resolution. It is contemplated in the invention described herein, that a combination of these technologies is feasible coupled with a detection system that allows the driver to adjust the sensitivity and thus the resolution of the capacitance system.
Resistance sensitive systems are also applicable and may also require the resolution adjustment system to account for people wearing heavy gloves.
Both the capacitance and resistance systems described in the above patents and publications usually have at least one rigid surface that forms the touch pad base or support. For applications on the center of the steering wheel, provision must be made for the airbag cover to open unimpeded by either the mass or strength of the touch pad. This is a different set of requirements than experienced in any of the prior art. This requires, for example, with the use of the capacitive system, that thin flexible circuits be used in place of rigid printed circuit boards. In the case of the resistive system, thin resistive pressure sensitive inks will generally be used in place of thicker variable resistance pads. Thin metal oxide films on thin plastic films can also be used, however, the durability of this system can be a serious problem.
Force sensing systems also require that the member upon which the force is applied be relatively rigid so that the force is transmitted to the edges of the touch pad where strain gages are located or where the supporting force can be measured. This requirement may also be incompatible with an airbag deployment doors unless the pad is placed wholly on one flap of the deployment door or multiple pads are used each on a single flap.
The use of a thin piezoelectric polymer film, especially in a finger tapping switch actuation mode, is feasible where the electrical resistance of the film can be controlled and where the signal strength resulting from a finger tap can be measured at the four corners of the touch path. Aside from this possible design, and designs using a matrix or tube structure described below, it is unlikely that surface acoustic wave or other ultrasonic systems will be applicable.
It should be noted that the capacitive touch pad is a technology of choice primarily because of its high resolution in the glove-less mode and the fact that it requires a very light touch to activate.
Although the discussion here has concentrated on the use of touch pad technologies, there are other input technologies that may be usable in some particular applications. In particular, in addition to the touch pad, it will be frequently desirable to place a variety of switches at various points outside of the sensitive area of the touch pad. These switches can be used in a general sense such as buttons that are now on a computer mouse, or they could have dedicated functions such as honking of the horn. Additionally functions of the switches can be set based on the screen that is displayed on the heads up display. A matrix of switches can of course replace the touch pad and they need not be places on the center of the steering wheel but could also be placed on the rim.
For some implementations, a trackball, joystick, button wheel, or other pointing device such as a gesture recognition system may be desirable. Thus, although the preferred embodiment of this invention contemplates using a capacitive or resistance touch pad as the input device, all other input devices, including a keyboard, could be used either in conjunction with the touch pad or, in some cases, as a replacement for the touch pad depending on the particular application or desires of the system designer.
Although the primary focus of this invention has been to develop a heads up display and interactive input devices for location on the steering wheel, in many cases it will be desirable to have other input devices of a similar nature located at other places within the vehicle. For example, an input device location for a passenger may be on the instrument panel, the armrest or attached in an extension and retraction arrangement from any surface of the passenger compartment including the seats, floor, instrument panel, headliner and door. In some cases, the device may be removable from a particular storage location and operated as a hand-held device by either the passenger or the driver. The interface thus can be by hard wire or wireless.
Voice recognition systems are now being applied more and more to vehicles. Such systems are frequently trained on the vehicle operator and can recognize a limited vocabulary sufficient to permit the operator to control many functions of the vehicle by using voice commands. These voice systems are not 100% accurate and heretofore there has been no effective means to provide feedback to the operator of the vehicle indicating what the voice system understood. When used with the heads up display interactive system described herein, a voice-input system can be used either separately or in conjunction with the touch pad systems described herein. In this case, for example, the vehicle operator would see displayed on the heads up display the results of voice commands. If the system misinterpreted the driver's command than a correction can be issued and the process repeated. For example, let us say that the vehicle operator gave a command to the vehicle phone system to dial a specific number. Let us assume that the system misunderstood one of the digits of the number. Without feedback, the driver may not know that he had dialed a wrong number. With feedback he would see the number as it is being dialed displayed on the heads up display and if he or she sees that an error occurred, he or she can issue a command to correct the error. In this manner, the interactive heads up display can function along with a voice command data input system as well as the touch pad systems described herein. The display can also be used as feedback for a gesture recognition system.
The invention disclosed herein can also be used in conjunction with U.S. Pat. No. 5,829,782 which describes, among other things, the use of an occupant location system to find the approximate location of the mouth of a vehicle operator. Once the location of the mouth has been determined, a directional microphone can focus in on that location and thereby significantly improve the accuracy of voice command systems.
In a similar manner also as described in U.S. Pat. No. 5,822,707 the location of the driver's eyes can be approximately determined and either the seat can be adjusted to place the operators eyes into the eye ellipse, which would be the ideal location for viewing a heads up display or, alternately, the heads up display protection system can be adjusted based on the sensed location of the eyes of the occupant. Although several prior art patents have disclosed the capability of adjusting the heads up display, none of them have done so based on a determination of the location of the eyes of the occupant.
One of the problems with heads up displays as described in the prior art patents is that sometimes the intensity of light coming in from the environment makes it difficult to see the information on the heads up display. In U.S. Pat. No. 5,829,782, a filter is disclosed that can be placed between the eyes of the vehicle operator and the source of external light, headlights or sun, and the windshield can be darkened in an area to filter out the offending light. This concept can be carried further when used with a heads up display to darken the area of the windshield where the heads up display is mounted, or even darken the entire windshield, in order to maintain a sufficient contrast ratio between the light coming from the automatically adjusted heads up display optical system and the light coming from the real world scene. This darkening can be accomplished using electrochromic glass or a liquid crystal system.
An alternate method of communicating with a touch pad or other input device is to do so by passive wireless means. In one implementation of this approach in accordance with the invention, a multi-loop cable is placed around the vehicle and used to inductively charge a circuit located on the touch pad or other input device. The device itself can these be totally free of wires since the information that it sends can also be transmitted wireless to the loop, which now acts as an antenna. The device can now be placed anywhere in the vehicle and in fact it can be moved from place to place without concern for wires.
Although the invention described herein is not limited to a particular heads up display technology, the preferred or best mode technology is to use the garnet crystal heads up system described in U.S. Pat. No. 5,473,466. Although the system has never been applied to automobiles, it has significant advantages over other systems particularly in the resolution and optical intensity areas. The resolution of the garnet crystals as manufactured by Revtek is approximately 600 by 600 pixels. The size of the crystal is typically 1 cm square. Using a laser protection system, a sufficiently large heads up display can be obtained while the system occupies a volume considerably smaller than any system described the prior art. By using a monochromatic laser as the optical source, the energy absorbed by the garnet crystal is kept to a minimum.
An alternate technology that can be used for a heads-up display is based on OLED (organic light emitting diode) technology whereby the projection system is no longer needed and a film that can be sandwiched between the sheets of glass that make up the windshield can be made to emit light. Naturally other locations for the OLED can be used.
These patents are meant to be representative of prior art and not exhaustive. Many other patents that make up the prior art are referenced by the patents reference herein.
All prior art touch systems are active continuously. In the system of this invention, it is contemplated that the heads up display system may only be active or visible when in use. There is no known combination of the prior art that is applicable to this invention.
As the number of functions which the operator must perform while driving the vehicle is increasing, there is a need for a system which will permit the operator to perform various functions related to operating other vehicle systems without requiring him or her to take his or her eyes off of the road.
Such a system will not add undue additional burden to the driver. On the contrary, it will lessen the work load since the driver will not need to take his or her eyes off of the road to control many functions now being performed. On the same basis that people can read road signs while they are driving, people will not have a problem reading messages that are displayed on the heads-up display with the focal point out in front of the vehicle while they are driving, as long as the messages are kept simple. More complicated messages become possible when vehicles are autonomously driven.
5.0 Definitions
As used herein, a diagnosis of the “state of the vehicle” means a diagnosis of the condition of the vehicle with respect to its stability and proper running and operating condition. Thus, the state of the vehicle could be normal when the vehicle is operating properly on a highway or abnormal when, for example, the vehicle is experiencing excessive angular inclination (e.g., two wheels are off the ground and the vehicle is about to rollover), the vehicle is experiencing a crash, the vehicle is skidding, and other similar situations. A diagnosis of the state of the vehicle could also be an indication that one of the parts of the vehicle, e.g., a component, system or subsystem, is operating abnormally.
As used herein, a “part” of the vehicle includes any component, sensor, system or subsystem of the vehicle such as the steering system, braking system, throttle system, navigation system, airbag system, seatbelt retractor, air bag inflation valve, air bag inflation controller and airbag vent valve, as well as those listed below in the definitions of “component” and “sensor”.
As used herein, a “sensor system” includes any of the sensors listed below in the definition of “sensor” as well as any type of component or assembly of components which detect, sense or measure something.
The term “vehicle” shall mean any means for transporting or carrying something including automobiles, trucks, vans, containers, trailers, boats, railroad cars and engines.
The term “gage” as used herein interchangeably with the terms “gauge”, “sensor” and “sensing device”.
The following additional terms will be used in the description of the invention and for the sake of clarity are defined here.
The “A-pillar” of a vehicle and specifically of an automobile is defined as the first roof supporting pillar from the front of the vehicle and usually supports the front door. It is also known as the hinge pillar.
The “B-Pillar” is the next roof support pillar rearward from the A-Pillar.
The “C-Pillar” is the final roof support usually at or behind the rear seats.
The windshield header as used herein includes the space above the front windshield including the first few inches of the roof. The headliner is the roof interior cover that extends back from the header.
The term “squib” represents the entire class of electrically initiated pyrotechnic devices capable of releasing sufficient energy to cause a vehicle window to break, for example. It is also used to represent the mechanism which starts the burning of an initiator which in turn ignites the propellant within an inflator.
The term “airbag module” generally connotes a unit having at least one airbag, gas generator means for producing a gas, attachment or coupling means for attaching the airbag(s) to and in fluid communication with the gas generator means so that gas is directed from the gas generator means into the airbag(s) to inflate the same, initiation means for initiating the gas generator means in response to a crash of the vehicle for which deployment of the airbag is desired and means for attaching or connecting the unit to the vehicle in a position in which the deploying airbag(s) will be effective in the passenger compartment of the vehicle. In the instant invention, the airbag module may also include occupant sensing components, diagnostic and power supply electronics and componentry which are either within or proximate to the module housing.
The term “occupant protection device” or “occupant restraint device” as used herein generally includes any type of device which is deployable in the event of a crash involving the vehicle for the purpose of protecting an occupant from the effects of the crash and/or minimizing the potential injury to the occupant. Occupant restraint or protection devices thus include frontal airbags, side airbags, seatbelt tensioners, knee bolsters, side curtain airbags, externally deployable airbags and the like.
“Pattern recognition” as used herein will generally mean any system which processes a signal that is generated by an object (e.g., representative of a pattern of returned or received impulses, waves or other physical property specific to and/or characteristic of and/or representative of that object) or is modified by interacting with an object, in order to determine to which one of a set of classes that the object belongs. Such a system might determine only that the object is or is not a member of one specified class, or it might attempt to assign the object to one of a larger set of specified classes, or find that it is not a member of any of the classes in the set. The signals processed are generally a series of electrical signals coming from transducers that are sensitive to acoustic (ultrasonic) or electromagnetic radiation (e.g., visible light, infrared radiation, capacitance or electric and/or magnetic fields), although other sources of information are frequently included. Pattern recognition systems generally involve the creation of a set of rules that permit the pattern to be recognized. These rules can be created by fuzzy logic systems, statistical correlations, or through sensor fusion methodologies as well as by trained pattern recognition systems such as neural networks, combination neural networks, cellular neural networks or support vector machines.
A trainable or a trained pattern recognition system as used herein generally means a pattern recognition system that is taught to recognize various patterns constituted within the signals by subjecting the system to a variety of examples. The most successful such system is the neural network used either singly or as a combination of neural networks. Thus, to generate the pattern recognition algorithm, test data is first obtained which constitutes a plurality of sets of returned waves, or wave patterns, or other information radiated or obtained from an object (or from the space in which the object will be situated in the passenger compartment, i.e., the space above the seat) and an indication of the identify of that object. A number of different objects are tested to obtain the unique patterns from each object. As such, the algorithm is generated, and stored in a computer processor, and which can later be applied to provide the identity of an object based on the wave pattern being received during use by a receiver connected to the processor and other information. For the purposes here, the identity of an object sometimes applies to not only the object itself but also to its location and/or orientation in the passenger compartment. For example, a rear facing child seat is a different object than a forward facing child seat and an out-of-position adult can be a different object than a normally seated adult. Not all pattern recognition systems are trained systems and not all trained systems are neural networks. Other pattern recognition systems are based on fuzzy logic, sensor fusion, Kalman filters, correlation as well as linear and non-linear regression. Still other pattern recognition systems are hybrids of more than one system such as neural-fuzzy systems.
The use of pattern recognition, or more particularly how it is used, is important to the instant invention. In the above-cited prior art, except in that assigned to the current assignee, pattern recognition which is based on training, as exemplified through the use of neural networks, is not mentioned for use in monitoring the interior passenger compartment or exterior environments of the vehicle in all of the aspects of the invention disclosed herein. Thus, the methods used to adapt such systems to a vehicle are also not mentioned.
A pattern recognition algorithm will thus generally mean an algorithm applying or obtained using any type of pattern recognition system, e.g., a neural network, sensor fusion, fuzzy logic, etc.
To “identify” as used herein will generally mean to determine that the object belongs to a particular set or class. The class may be one containing, for example, all rear facing child seats, one containing all human occupants, or all human occupants not sitting in a rear facing child seat, or all humans in a certain height or weight range depending on the purpose of the system. In the case where a particular person is to be recognized, the set or class will contain only a single element, i.e., the person to be recognized.
A “combination neural network” as used herein will generally apply to any combination of two or more neural networks that are either connected together or that analyze all or a portion of the input data. A combination neural network can be used to divide up tasks in solving a particular occupant problem. For example, one neural network can be used to identify an object occupying a passenger compartment of an automobile and a second neural network can be used to determine the position of the object or its location with respect to the airbag, for example, within the passenger compartment. In another case, one neural network can be used merely to determine whether the data is similar to data upon which a main neural network has been trained or whether there is something radically different about this data and therefore that the data should not be analyzed. Combination neural networks can sometimes be implemented as cellular neural networks.
Preferred embodiments of the invention are described below and unless specifically noted, it is the applicants' intention that the words and phrases in the specification and claims be given the ordinary and accustomed meaning to those of ordinary skill in the applicable art(s). If the applicant intends any other meaning, he will specifically state he is applying a special meaning to a word or phrase.
Likewise, applicants' use of the word “function” here is not intended to indicate that the applicants seek to invoke the special provisions of 35 U.S.C. §112, sixth paragraph, to define their invention. To the contrary, if applicants wish to invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, they will specifically set forth in the claims the phrases “means for” or “step for” and a function, without also reciting in that phrase any structure, material or act in support of the function. Moreover, even if applicants invoke the provisions of 35 U.S.C. §112, sixth paragraph, to define their invention, it is the applicants' intention that their inventions not be limited to the specific structure, material or acts that are described in the preferred embodiments herein. Rather, if applicants claim their inventions by specifically invoking the provisions of 35 U.S.C. §112, sixth paragraph, it is nonetheless their intention to cover and include any and all structure, materials or acts that perform the claimed function, along with any and all known or later developed equivalent structures, materials or acts for performing the claimed function.